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The qualitative case study approach was used to 
collect data about the impact cf the National Center fcr Research in 
Vocational Education's (NCRVE) work on teacher education and 
competency. Two exemplary sites using a set of performance-based 
teacher education (PBTE) modules and supporting materials developed 
by NCRVE were selected for detailed study of impact: Temple 
University and the University of Central Florida. Major focus of 
field work was determining the effects of the PBTE modules, findings 
indicated PBTE (1) helped vocational teacher education programs 
survive in a period of retrenchment; (2) increased access to 
vocational teacher certification, accountability, convenience to 
teachers, flexibility, teacher competency in some areas, and 
teacher's sel£-evaluation and responsibility for learning; (3) 
shortened time required to certify teachers: and (H) changed 
teacher's role from classroom lecturer to instructional manager. 
Conclusions drawn about effects of PBTE curricula included that it 
has changed aspects of delivery of vocational teacher education, 
appears more effective and efficient in educating teachers than 
traditional approaches, is perceived by users as a high quality 
product, provides impetus to the competency-based vocational 
education movement, and appears to have long-term impact on improving 
teachers. (XIE) 
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Is R&D bringing about significant changes in the fabric of vocational 
education? What's really different from our investment jr. voc ed R s D? 
These questions are ambitious but to fully understand the impact of research 
on voc ed, we need to go beyond numbers to some of the deeper questions. 

In this paper, I will examine some aspects of the deeper impact of 
research. I will concentrate primarily on the work of the National Center 
for Research in Vocational Education. During the past 3 years, several 
impact studies have been conducted tp examine the work on nationally signif- 
icant problems such as increasing sex 6quity\ improving evaluation,^ improving 
career guidance^ ' "^and improving teacher education.^ 

In examining these problems, we wanted to tell an upbeat story, not a 
biased story but a real story about aspects of research that had truly made 
a difference. It became apparent in the early conceptual phases that an 
experimental approacn focusing on quantitative data would produce disappointing 
results. Like many research organizations, the National .enter works on 
complex problems which are not easy to c! unge . Most of the vo.k is not 
focused on classrooms but on the broader networks and systeii- of vocational 
^ education. When effects do occur from research, they are often subtle, highly 

individualized in different settings, and evolving in an interactive pattern 
^ with many other forces. 
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Qualitative Methodology 
We decided to turn to an alternative approach that is receiving increased 
attention in educational circles, namely qualitative or naturalistic 
methodology. The purpose of this approach is discover , the effects of a 
program through observing it in action in its r. tural context. The variables 
of interest are not preselected but emerge as th. program is observed. The 
effect of a program is not reduced to a sinqular Phonomonor^ but rather a 
pluralistic view, representing multiple realities of the program is taken. 
Qualitative methodology provides a rich description of a program from the 
perspectives of che different people involved. 

The technique used to collect data about the impact of several National 
Center projects was qualitative case studies. Projects were studied on site 
so that their effects could be fully understood. An interview schedule was 
developed which outlined major topics and sub-topics rather than a set of 
specific questions. Data was collected through in-depth open ended interviews 
with a variety of project staff and participants. Purposoful samples of extreme 
cases (advocates/adversaries; deeply involved/superficially involved) , typical 
cases, and key informants were used. Interviews were supplemented by obser- 
vation of the program and examination of various records. Tr iangulation , a 
procedure for checking patterns of similarity or difference across different 
data sources, was used to test the accuracy of the information collected. 
A site visit team comprised of tliree evaluators, one member from outside 
the National Cent ■, was used to enhance credibility and facilitate triangu- 
lation among multiple observers. 

During the interviews, questions were asked in ways that allowed 
respondents to direct the interview to the issues that were most important 
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to them, such as "What have you learned from this progr m?" and "What do you 

do now that you didn't do before the program began?" Respondents also 

were encouraged to describe the program in their own words through questions, 

such as "Take me through a typical day in " or "What would I see in 

terms of a participant's progress in ". Emphasis was placed on 

soliciting direct quotes and recording them verbatim. Quotes allowed many 
subtle findings to be maintained and were quite insightful during analysis. 

The Impact of the 
National Center's Work on 
Teacher Eucation 

Rather than superficially discussing impact on a number of problems, 
I will focus on the problem of teacher education and competency. After 
examining the impact on this problem, some of the broader issues related to 
impact on problems of national significance will be discussed. By far the 
most widespread and deepest impact of the National Center can be found in 
the area of teacher education. After 10 years of work ind approximately 
two million dollars of federal support, the National Center has played a 
key role in the growth of performance-based teacher education (PBTE) . 
Beginning with research on teacher competencies, the National Center developed 
a set of 100 PBTE^' modules and supporting materials, tested these materials, 
and conducted a wide range of training activities. Two exemplary sites using 
PBTE were selected for detailed study of impact: Temple University in 
Philadelphia, Pennsylvania and the University of Central Florida in Orlando, 
Florida. Additional data were collected through telephone interviews and 
.surveys of 71 different teacher education programs. 

*The term PBTE is used in this study to refer to the National Center's 
PBTE curriculum rather than to the general concept of PBTE. 



Effects of PBTE 

Determining- the effects of the PBTE modules was the major focus of the 
field work. In interviews questions were asked about the specific effects 
and consequences of using the modules, as well as the more general effects 
of performance-based teacher education programs. This sectiori reviews the 
findings on effects. 

!• Helped vocational teacher education programs survive in d period 
of retrenchment . It is clear from a number of interviews that when the PBTE 
program and modules were introduced, a potential for a seriour decline of 
vocational teacher education was present at one of the universities studied. 
Moreover the changing funding emphasis at the state level, changing demands 
of vocational schools and teachers in those schools, and possible university 
retrenr hment combined to pose a serious threat to the continuation of voca- 
tional teacher education. The development of a PBTE program--with its 
field emphasis, utili.zation of professionally developed modules, and pattern 
of differentiated staffing — made it possible to receive state funds for 
program development and to maintain student participation in the program 
in a period of decline. 

2 • Increased access to vocational teacher cerrif ication . Teacher 
education programs which have adopted PBTE are now able to serve a larger 
geographic area with small groups of teachers wherever they happen to be. 
Individualized instruction can be developed through the modules wherever 
training is needed. The chairman of one department explained this change 
as follows: 

Previotasly we were having the problem of delivering courses. That's 
the main thing this new program has done. We can deliver the [urogram 
at any time right where the student needs and wants it. There's no 
question in my mind that the big thing is delivering the right 
ini?truction to the right people at the right time. 
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^- Shortened the time required to certify teachers . In one institution*, 
faculty, staff, and students felt there has been a substantial reduction 
in the time it takes to obtain vocational certification. Before the 
introduction of performance-based teacher education and PBTE modules, it 
often took students three years to complete the program for certification; 
now most students complete the program in one and a half years. As one 
director put it: 

One of the good things about the program is its productivity, the 
ability to move people through the program, complete the modules, 
and move on... The research evidence says that a classroom 
approach and our approach produce basically the same results in 
teachers, but our approach takes considerably less time. I 
call that a major improvement. 

^' Increased accountability . The administration and faculty at the 

university believe that the total impact of the new program is to make the 

university more accountable. Through the introduction of PBTE modules, the 

curriculum has been standardized and made explicit so that the content of 

vocational teacher education is no longer subject to the uncertainties of 

the interests and orientation of which ever professor happens to be teaching 

a particular course. One university administrator said: 

Our program could not exist without the modules ... the materials have 
been proven. That's one of the real strengths of these modules. 
When you get through them, you can do what they've taken you 
through. .. it ' s a more accountable way of operating. There are 
not many programs that know what their students can do when they're 
done. We do know. We've looked at it. We know our products--what 
they can do. 

5- Changed teacher educators role from a classroom lecturer to 
instructional manager. While teacher educators have had a major impact on 
the implementation of the program, they have also been affected by the 
program. Increased contact with the community and more personalized and 



individualized contact with teachers in the program are two changes in role 
responsibility. Less time is spent in lecture and formal instruction. 
Faculty autonomy in determining course content has been diminished and 
replaced with an increased emphasis on managing instruction. Most 
teacher educators view this change positively as summarized by a program 
director: 

The reaction of teacher educators has been extremely positive. 
They didn't like lecturing much. They feel they're more 
effective now. There is no ff^eling of loss in not deter-nining 
the full content of the curriculum. I think they're more 
satisfied. 

6. Increased convenience to teachers . In interviews with teachers, 
this theme of convenience consistently emerged. An occupational health 
instructor said: 

I take the modules at a slow pace because I have too many hats 
to wear already. The modules have permitted me to do it when I 
have time instead of fitting my schedule to a lot of class time. 
I work at my own pace. 

Another student said: 

I think being able to choose my own sequence and own schedule was 
the most important thing to me. This really b»^ats the hell out of 
college lecturers reading to us out of a textbook. 

7. I ncreased flexibility . Because the program is individualized, 

inservice teachers can begin the program whenever they are hired. Teachers 

hired in the middle of the year, or even in the middle of a quarter, can 

sign up immediately. As a director of the program put it: 

We can now give teachers help immediately, whenever they are hired. 
We can only do that with modules in an individualized program. 

8. Increased teacher competency in some areas of instruction . A 
major focus of the interviews was on the long term effects of participating 
in the program. The issue here is important: having demonstrated a 
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competency at a given point in time, does that competency carry over and 
manifest itself in actual teaching situations or is the competency lost 
or ignored once the requirements for certification have been fulfilled: 
We put this question to faculty, administrators and teachers who had 
completed the program. Many of the teachers reported they used what they had 
learned without having to think about it. There were, however, some 
common elements that were noted as having long term carry-over effects in 
teaching : 

o An increased consciousness about objectives — the importance of 
having objectives and making them explicit to students 

o More polished skills in developing lesson plans and organizing 
content into units of instruction 

o Increased awareness o£ the importance of using reinforcement 
techniques in teaching 

o Improved skills in individualising instruction 

o Improved skills in evaluating the performance of their students 

One teacher interviewed said that her teaching had been substantially 

affected by her experience in the PBTE program: 

About every module I've taken has been really relevant to my 
teaching situation. I * ve been able to develop many things I use 
from the modules. I now teach in what I call a modified 
competency-based approacn. I use lab handouts, actually mini- 
modules that I developed myself. I evaluate my students when 
they're ready. They work at their own pace. The mini-modules 
work really well. My teaching is completely organized around 
their independent competency work. 

It is clear that the issue of long-term effects is an important one. It 
is also clear the.t the data presented here about the long-term effects in 
teacher competencies is no more than impressionistic. The effects 
identified here suggest important outcomes that should be studied over time 
in a comparative framework to fully understand the effects of performance- 
based teacher education. 

ERIC 



m^-^-ed teacher's self-evaluation and r..n.ne. . . ... 

learning. a common theme expressed by p.ogran, staff was the extent to 
Which program participants learned how to be self -evaluative. In order to 
successfully get through the modules, teachers had to learn how to assess 
their own work and competencies. That process of self -assessment became 
internalized and carried over to affect their teaching in the long run. 

Related to the theme of se If -assessment is the idea of "taking 
responsibility for one's own learning." . program director expressed 
the importance of this idea and its effects on the learning process as 



follows : 



take'respoisibM°:^ '''^ 1— ed to 

take responsibility tor his own learning-it takes a whii. i 

understand that the modules are rolated'to whaf they are doina 

as teachers Once they understand this and begin to take ' 

responsibility for their own learning they can go ^hrouah 

the'v" e r '"'"' ''^^ ^'^^ reLvant'to lit 

they are doing out there in their classroom. 

Weaknesses of PBTE 

Although this study focused pri„„ily ^^^^^^^^^ 
PBTE, the„ was evidence of so.e opposition to the pe.to™a„ce-based eduction 
and parfo^ance-based teacher education .ove.ehts i„ the institutions studied. 
The opposition was typically expressed as opposition toward the concept of 
perfor..nce-based education which is viewed as overly mechanized and 
Simplistic, rather than toward the PBTE modules. So.e direct quotes about 
perceived weaknesses of PBTE are presented below to preserve the flavor of 
the interviews. 

an atJ;mp??S^;..;K::iL'1:^^r -en sacrificed in 

a single document can be reL^ant to tlT"" Population. There's no way 

levels." (Teacher educator) '° everything for all 
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2. Reduce individuality . "Education is both an art and a science. PBTE 
is individualized but it doesn't permit individuality; it actually reduces 

the latter." (Teacher educator) 

3. Limited emphasis on affective domain . "The modules have a tendency to 
develop 'technicians* rather than professionals. A module explains how to 
develop a lesson plan rather than describing how a lesson plan fits into a 
total education program. The inductive, thoughtful, and interactive approach 
to learning is diminished. Developing an educational philosophy is more a 
dialectice process than a process of programmed instruction through modules." 
(Teacher educator) 

4. Require highly motivated students . "PBTE requires students to bo 
relatively highly motivated and self-directed. Some students fall through the 
cracks and don't really get the attention that they need to complete the program.' 
(Teacher educator) 

5. Potential for misuse . "Some faculty see the modules as an easy way out 
of teaching." (Teacher educator) 

"Now I just do the assignment. I s+-art at the back of the book and then 
skim the module. No one's going to quiz you to see what you know." (Teacher) 

"Modules are misused by administrators who think modules are 'band-aids' 
to slap onto a teaching problem." (Teacher educator) 

Conclusions 

Although there is some evidence of opposition to the competency-based 
education and PBTE movements in some of the institutions studied, the following 
conplusions can be drawn about the effects of the PBTE curricula: (X) most 
users express a strong feeling of ownership of their PBTE programs; (2) it 
has changed many aspects of the delivery of vocational teacher education; 
(3) it appears to be more effective and efficient in educating teachers than 
traditional approaches; (4) it is perceived by users as a hiqh quality 
product; (5) it is providing impetus to the movement toward competency-based 
instruction for all vocational education students; and (6) it appears to 
be having a long-term impact on improving vocational education teachers. 
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Factors That Enhance Impact of Research 
In this paper, teacher education was portrayed as an example of a 
nationally significant problem that .-as addressed by substantial R ^ D effort. 
In comparing the findings from this study with other studies of impact on 
nationally significant problems, several factors that enhance the impact of 
R S D emerged as important. At first glance these factors may seem obvious. 
But since they are frequently absent in R & D efforts, ,t is important to 
take a second glance. No systematic attempt to examine the relative importance 
of these factors has been undertaken. But based on expo - ) • .;.c,^.s -...i t h and 
reflections on the products, the following critical factors for success of 
an R & D effort are offered. A successful R&D effort would: 



1, 



Meet a truly critical need - one that appears to be increasing, 
one that is not being well met by other products in use or 
under development, and one that it makes sense to meet 
through an R & D effort. 

Have a clearly defined target audience which is involved in 
developing, testing, and using the product as early and 
as much as possible. 

Have a committed and highly persistent project staff who care 
about their work and become personal advocates for its use 
in the field. 

Use considerable and varied dissemination techniques including 
workshops, technical assistance, deploying former project 
staff to start field efforts, establishing networks and 
consortiums of users, and using successful users/sites to 
train others. 

Provide a product Lhat is multifaceted and floxibl- .r. thai 
It has appeal to different users in varied settings. The 
product should allow users to adapt the product to their 
own settings. This feature facilitates ownership which, 
m turn, facilitates continued use of the product. 

Provide a high quality, carefully researched, fully tested, and 
well written product. 
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7. Provide a product that is used with or has direct impact on 

students. (This refers to students at all levels of education.) 
Since students are the main business of education, this 
feature helps assure that the product will be used. 



8. Provide professionally printed products to insure there will be 
sufficient quantities reaoily available to users when reeded. 

9. Have products which are as short as possible or divided into 
short modules. Products should contain executive summaries ^ inde 
detailed table of contents and other tools to mako their use 
convenient. 
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